Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid

Naive-Bayes induction algorithms were previously shown to be surprisingly accurate on many classii-cation tasks even when the conditional independence assumption on which they are based is violated. However , most studies were done on small databases. We show that in some larger databases, the accuracy of Naive-Bayes does not scale up as well as decision trees. We then propose a new algorithm, NBTree, which induces a hybrid of decision-tree classiiers and Naive-Bayes classiiers: the decision-tree nodes contain uni-variate splits as regular decision-trees, but the leaves contain Naive-Bayesian classiiers. The approach retains the interpretability of Naive-Bayes and decision trees, while resulting in classiiers that frequently out-perform both constituents, especially in the larger databases tested.